{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 协程的本质\n",
    "- 协程的调度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.1 为什么要有协程，线程不好用吗？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 进程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 操作系统 “程序” 的最小单位\n",
    "- 进程用来占用内存空间\n",
    "- 进程相当于厂房，占用工厂空间"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 每个进程可以有多个线程\n",
    "- 线程使用系统分配给进程的内存，线程之间共享内存"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 线程用来占用CPU时间\n",
    "- 线程的调度需要由系统进行，开销较大\n",
    "- 线程相当于工厂的生产线，占用工人的工时"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线程的问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 线程本身占用资源大\n",
    "- 线程的操作开销大\n",
    "- 线程切换开销大"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 协程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 协程就是将一段程序的运行状态打包，可以在线程之间调度\n",
    "- 将生产流程打包，使得流程不固定在生产线上\n",
    "- 协程并不取代线程，协程也要在线程上运行\n",
    "- 线程是协程的资源，协程使用线程这个资源"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 协程的优势"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 资源利用\n",
    "- 快速调度\n",
    "- 超高并发"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.2 协程的本质是什么？*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "do3\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "signal: interrupt\n"
     ]
    }
   ],
   "source": [
    "import \"fmt\"\n",
    "\n",
    "func do1() {\n",
    "\tdo2()\n",
    "}\n",
    "\n",
    "func do2() {\n",
    "\tdo3()\n",
    "}\n",
    "\n",
    "func do3() {\n",
    "\tfmt.Println(\"do3\")\n",
    "}\n",
    "\n",
    "func main() {\n",
    "\tgo do1()\n",
    "\n",
    "\ttime.Sleep(time.Hour)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 调试: `5/5.1/main.go`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- https://go.dev/src/runtime/runtime2.go\n",
    "\n",
    "```go\n",
    "type g struct {\n",
    "  // ...\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 协程的底层结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- runtime 中，协程的本质是一个 `g` 结构体\n",
    "- `stack`：堆栈地址\n",
    "- `gobuf`：当前程序运行现场\n",
    "- `atomicstatus`：协程状态"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线程的抽象"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- runtime 中将操作系统线程抽象为 `m` 结构体\n",
    "- `g0`：g0协程，操作调度器\n",
    "- `curg`：`current g`，目前线程运行的`g`\n",
    "- `mOS`：操作系统线程信息"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- https://go.dev/src/runtime/runtime2.go\n",
    "\n",
    "```go\n",
    "type m struct {\n",
    "  // ...\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.3 协程是如何执行的？一 *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 线程循环\n",
    "\n",
    "<img src=\"./images/5.3.1.jpeg\" alt=\"5.3.1\" style=\"zoom:50%;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.4 协程是如何执行的？二"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 单线程循环"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 多线程循环"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/5.4.1.png\" alt=\"5.4.1\" style=\"zoom:50%;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 协程队列需要加锁"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线程循环"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 操作系统并不知道Goroutine的存在\n",
    "- 操作系统线程执行一个调度循环，顺序执行Goroutine\n",
    "- 调度循环非常像线程池"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 协程顺序执行，无法并发\n",
    "- 多线程并发时，会抢夺协程队列的全局锁"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.5 为什么要有G M P调度模型？*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 多线程并发时，会抢夺协程队列的全局锁"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 多线程循环"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/5.5.1.png\" alt=\"5.5.1\" style=\"zoom:50%;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 本地队列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/5.5.2.png\" alt=\"5.5.2\" style=\"zoom:40%;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## p结构体"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- https://go.dev/src/runtime/runtime2.go\n",
    "\n",
    "```go\n",
    "type p struct { \n",
    "  // ...\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/5.5.3.png\" alt=\"5.5.3\" style=\"zoom:40%;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## GMP模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/5.5.4.png\" alt=\"5.5.4\" style=\"zoom:40%;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"./images/5.5.5.png\" alt=\"5.5.5\" style=\"zoom:40%;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## P的作用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- M与G之间的中介（送料器）\n",
    "- P持有一些G，使得每次获取G的时候不用从全局找\n",
    "- 大大减少了并发冲突的情况"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- https://go.dev/src/runtime/proc.go"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 任务窃取"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 窃取式工作分配机制"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 如果在本地或者全局队列中都找不到G\n",
    "- 去别的P中“偷”\n",
    "- 增强了线程的利用率"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 新建协程"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 随机寻找一个P\n",
    "- 将新协程放入P的 runnext（插队）\n",
    "- 若P本地队列满，放入全局队列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.6 如何实现协程并发？*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 协程顺序执行，无法并发"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 协程饥饿问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线程循环"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线程循环(触发切换)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 全局队列饥饿问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 切换时机"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 主动挂起(runtime.gopark)\n",
    "- 系统调用完成时"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 主动挂起(runtime.gopark)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 系统调用完成时"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 如果协程顺序执行，会有饥饿问题\n",
    "- 协程执行中间，将协程挂起，执行其他协程\n",
    "- 完成系统调用时挂起，也可以主动挂起\n",
    "- 防止全局队列饥饿，本地队列随机抽取全局队列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 永远都不主动挂起\n",
    "- 永远都不系统调用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.7 抢占式调度解决了什么问题？*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 思路"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 有没有一个地方，经常会被调用？\n",
    "- `runtime.morestack()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## runtime.morestack()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `morestack()`的本意是检查协程栈是否有足够空间\n",
    "- 调用方法时，会被编译器插入`morestack()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 标记占用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 系统监控到 Goroutine 运行超过 10ms\n",
    "- 将 `g.stackguard`0 置为 `0xfffffade`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 抢占"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 执行`morestack()`时判断是否被抢占\n",
    "- 如果被抢占，回到`schedule()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 基于协作的抢占式调度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线程信号"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 操作系统中，有很多基于信号的底层通信方式\n",
    "- 比如 `SIGPIPE` / `SIGURG` / `SIGHUP`\n",
    "- 线程可以注册对应信号的处理函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 基于信号的抢占式调度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 注册 `SIGURG` 信号的处理函数\n",
    "- GC工作时，向目标线程发送信号\n",
    "- 线程收到信号，触发调度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 基于系统调用和主动挂起，协程可能无法调度\n",
    "- 基于协作的抢占式调度：业务主动调用 `morestack()`\n",
    "- 基于信号的抢占式调度：强制线程调用 `doSigPreempt()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.8 实战 协程太多有什么问题？"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 无线开启协程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "// 别点运行, 会卡死\n",
    "\n",
    "func main() {\n",
    "    for i := 0; i < math.MaxInt32; i++ {\n",
    "        go func(i int) {\n",
    "            fmt.Println(i)\n",
    "            time.Sleep(time.Second)\n",
    "        }(i)\n",
    "    }\n",
    "    time.Sleep(time.Hour)\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 协程太多的问题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 文件打开数限制\n",
    "- 内存限制\n",
    "- 调度开销过大"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 处理协程太多的方案"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 优化业务逻辑\n",
    "- 利用 channel 的缓存区\n",
    "- 协程池\n",
    "- 调整系统资源"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 利用 channel 的缓存区"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 利用channel的缓存机制\n",
    "- 启动协程前，向channel送入一个空结构体\n",
    "- 协程结束，取出一个空结构体"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 协程池(tunny)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 预创建一定数量的协程\n",
    "- 将任务送入协程池队列\n",
    "- 协程池不断取出可用协程，执行任务"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 协程池"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 慎用协程池"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Go语言的线程，已经相当于池化了\n",
    "- 二级池化会增加系统复杂度\n",
    "- Go语言的初衷是希望协程即用即毁，不要池化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.9 本章小结"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Go (gonb)",
   "language": "go",
   "name": "gonb"
  },
  "language_info": {
   "codemirror_mode": "",
   "file_extension": ".go",
   "mimetype": "text/x-go",
   "name": "go",
   "nbconvert_exporter": "",
   "pygments_lexer": "",
   "version": "go1.24.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
